Communication-minimal partitioning and data alignment of BLAS-like algorithms

Automatic data alignment and computation domain partitioning techniques have been widely investigated to reduce communication overheads in distributed memory systems. This paper considers how these two techniques can be combined and applied to BLAS-like algorithms. Current data alignment techniques focus on individual data entries and cannot be directly used for cases when blocks of entries should be aligned collectively. This paper shows that existing data alignment techniques can be applied to partitioned algorithms if the null space of the data array indexing matrix is a boundary of computation blocks or the intersection of some of the computation block boundaries. These conditions can be used to generate several diierent partitionings and time-space transformations from which the optimal ones can be chosen for a given target architecture. An example illustrates how it is possible to trade oo the number of communications and memory space. Another example shows partitions of matrix-matrix multiplication that have smaller communication-computation ratios than Cannon's algorithm does.